Papers with text summarization
Towards Opinion Summarization of Customer Reviews (P18-3)
Copied to clipboard
| Challenge: | Existing methods to summarize text are limited to small, homogeneous datasets . authors outline future directions to solve these problems . |
| Approach: | They propose to use neural networks to generate summaries of user-generated travel reviews . they aim to take into account shifting opinions over time and address these issues . |
| Outcome: | The proposed method will make it easier for users of review sites to make more informed decisions. |
Automatic and Human-AI Interactive Text Generation (with a focus on Text Simplification and Revision) (2024.acl-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we focus on text-to-text generation, a class of natural language generation tasks, that takes a piece of text as input and then generates a revision that is improved according to some specific criteria. |
| Approach: | This tutorial focuses on text-to-text generation, a class of natural language generation tasks that takes a piece of text as input and generates a revision that is improved according to some specific criteria. |
| Outcome: | This tutorial focuses on text-to-text generation, a class of natural language generation tasks, that takes a piece of text as input and generates a revision that is improved according to some specificcriteria. |
Discourse Analysis and Its Applications (P19-4)
Copied to clipboard
| Challenge: | Discourse processing is a suite of NLP tasks to uncover linguistic structures from texts at several levels, which can support many downstream applications. |
| Approach: | They present a set of tasks to uncover linguistic structures from texts at several levels, which can support many downstream applications. |
| Outcome: | The tutorial covers the basic concepts of discourse analysis and linguistic structures in monologue vs. conversation, synchronous v. asynchronous conversation, and key linguistic structure in discourse analysis. |
Exploiting Summarization Data to Help Text Simplification (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing text simplification datasets are limited to Wikipedia and Newsela, restricting further development of this field. |
| Approach: | They propose an alignment algorithm to extract sentence pairs from summarization datasets and a method to filter suitable pairs. |
| Outcome: | The proposed algorithm can extract sentence pairs from summarization datasets and perform well with real datasets. |
Creative Natural Language Generation (2023.emnlp-tutorial)
Copied to clipboard
| Challenge: | This tutorial aims to bring awareness of the important and emerging research area of open-domain creative generation. |
| Approach: | They will review recent studies on creative language generation at sentence level as well as longer forms of text. |
| Outcome: | This paper reviews recent studies on creative language generation at sentence level as well as longer forms of text. |
CDD: A Large Scale Dataset for Legal Intelligence Research (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Recent research has focused on predicting crimes, predicting outcomes of judicial debates, and extracting information from legal documents. |
| Approach: | They propose to use a large-size Court Debate Dataset to analyze court debates . they invite experienced judges to design appropriate labels for data records . |
| Outcome: | The proposed dataset includes 30,481 court cases, totaling 1,144,425 utterances. |
Evaluating Factual Consistency of Texts with Semantic Role Labeling (2023.starsem-1)
Copied to clipboard
| Challenge: | Existing evaluation methods rely on task-specific language models, which in turn hampers interpretation of generated scores. |
| Approach: | They propose a reference-free evaluation metric for text summarization that measures factuality . their method generates fact tuples from Semantic Role Labels, applied to both input and summary texts. |
| Outcome: | The proposed evaluation metric is comparable with state-of-the-art methods and has a stable generalization across datasets. |
Factual Consistency Evaluation for Text Summarization via Counterfactual Estimation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to evaluate factual consistency in text summarization neglect the intrinsic cause of factual inconsistency or rely on auxiliary tasks. |
| Approach: | They propose a method to evaluate the factual consistency in text summarization via counterfactual estimation, which formulates the causal relationship between source document, generated summary, and the language prior. |
| Outcome: | The proposed metric improves correlation with human judgments and convenience of usage on three public abstractive text summarization datasets. |
Annobot: Platform for Annotating and Creating Datasets through Conversation with a Chatbot (2020.coling-demos)
Copied to clipboard
| Challenge: | Using conversation with a chatbot, we create annotating and creating datasets through conversation with an open-source platform called Annobot. |
| Approach: | They propose an open-source platform for annotating and creating datasets through conversation with a chatbot. |
| Outcome: | The proposed platform has a wide range of applications including data labelling for binary, multi-class/label classification tasks, preparing data for regression problems and creating sets for issues such as machine translation, question answering or text summarization. |
Non-Autoregressive Text Generation with Pre-trained Language Models (2021.eacl-main)
Copied to clipboard
| Challenge: | Autoregressive generation models generate tokens in a left-to-right, token-by-token fashion, resulting in lag in inference. |
| Approach: | They propose to use BERT as the backbone of a non-autoregressive generation model for greatly improved performance. |
| Outcome: | The proposed model outperforms existing non-autoregressive models and achieves competitive performance with many strong autoregressive model. |
Cool English: a Grammatical Error Correction System Based on Large Learner Corpora (C18-2)
Copied to clipboard
| Challenge: | Existing systems that correct grammatical errors are lacking in second language learning due to limited vocabulary and inadequate command of grammar. |
| Approach: | They propose a grammatical error correction system that provides corrective feedback for essays using a sequence-to-sequence model. |
| Outcome: | The proposed system achieves competitive performance on a number of publicly available testsets. |
SummVis: Interactive Visual Analysis of Models, Data, and Evaluation for Text Summarization (2021.acl-demo)
Copied to clipboard
| Challenge: | despite advances in abstractive text summarization, the true performance and failure modes of modern neural models are not yet fully understood due to the black-box nature of neural models and unmanageable scale of recent datasets for manual analysis. |
| Approach: | They propose an open-source tool for visualizing abstractive summaries that enables fine-grained analysis of models, data, and evaluation metrics associated with text summarization. |
| Outcome: | The proposed tool can identify the shortcomings and failure modes of state-of-the-art summarization models. |
A Hierarchical Network for Abstractive Meeting Summarization with Cross-Domain Pretraining (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods of summarizing meetings require complex multi-step pipelines that are intractable. |
| Approach: | They propose an abstractive summary network that adapts to meeting transcripts by hierarchical structure and role vectors. |
| Outcome: | The proposed model outperforms existing methods in both metrics and human evaluation. |
Automatic Spelling Correction for Resource-Scarce Languages using Deep Learning (P18-3)
Copied to clipboard
| Challenge: | Indic languages are resource-scarce and do not have such parallel data due to low volume of queries. |
| Approach: | They propose a sequence-to-sequence deep learning model which trains end-to end for Indic languages, Hindi and Telugu. |
| Outcome: | The proposed model is competitive with existing spell checking and correction techniques for Indic languages. |
SummEval: Re-evaluating Summarization Evaluation (2021.tacl-1)
Copied to clipboard
Alexander R. Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, Dragomir Radev
| Challenge: | a lack of comprehensive studies on evaluation metrics for text summarization hinders progress . a new study aims to improve evaluation metrics that correlate with human judgments . |
| Approach: | They propose to re-evaluate automatic evaluation metrics and share a toolkit for evaluation . they hope to promote a more complete evaluation protocol for text summarization . |
| Outcome: | The proposed evaluation metrics are inconsistent with existing evaluation protocols. |
On the Role of Summary Content Units in Text Summarization Evaluation (2024.naacl-short)
Copied to clipboard
Marcel Nawrath, Agnieszka Nowak, Tristan Ratz, Danilo Walenta, Juri Opitz, Leonardo Ribeiro, João Sedoc, Daniel Deutsch, Simon Mille, Yixin Liu, Sebastian Gehrmann, Lining Zhang, Saad Mahamood, Miruna Clinciu, Khyathi Chandu, Yufang Hou
| Challenge: | a human written summary content unit (SCU) is used to judge the quality of a summary . a pyramid evaluation method is based on SCUs that decompose a reference summary into concise sentences . |
| Approach: | They propose to use automated SCUs to evaluate the quality of a candidate summary . they propose to generate SCU approximations from AMR meaning representations and large language models . |
| Outcome: | The proposed method can be fully automated, but lacks the human effort to validate it. |
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets (2023.findings-acl)
Copied to clipboard
Md Tahmid Rahman Laskar, M Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan, Shafiq Joty, Jimmy Huang
| Challenge: | Currently, the evaluation of large language models (LLMs) such as ChatGPT in academic datasets is difficult due to the difficulty of evaluating the generative outputs produced by this model against the ground truth. |
| Approach: | They evaluate ChatGPT across 140 tasks and analyze 255K responses it generates in academic datasets. |
| Outcome: | The proposed model performs well on 140 tasks and generates 255K responses in these datasets. |
SumPubMed: Summarization Dataset of PubMed Scientific Articles (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing summarization models that can extract the top few lines of news articles fail to summarize long documents. |
| Approach: | They constructed a scientific summarization dataset from MEDLINE articles from the PubMed archive to address this problem. |
| Outcome: | The proposed model outperforms existing models on news article summarization datasets and shows that it is more efficient to extract the top few lines. |
Mapping the Design Space of Human-AI Interaction in Text Summarization (2022.naacl-main)
Copied to clipboard
| Challenge: | Automated text summarization systems involve humans for preparing data or evaluating model performance, yet, there is no systematic understanding of human-AI interactions and how to design for them. |
| Approach: | They conducted a systematic literature review of 70 papers and designed prototypes for each interaction. |
| Outcome: | The proposed design considerations were based on the results of a systematic literature review of 70 papers and interviews with 16 users. |
What Have We Achieved on Text Summarization? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for text summarization have been investigated, but there are still gaps between them and human professionals. |
| Approach: | They analyze 8 major sources of errors on 10 representative summarization models manually. |
| Outcome: | Aiming to gain more understanding of summarization systems with respect to their strengths and limitations on a fine-grained syntactic and semantic level, we use 8 major sources of errors on 10 representative summarizing models. |
An Exploration of Post-Editing Effectiveness in Text Summarization (2022.naacl-main)
Copied to clipboard
Vivian Lai, Alison Smith-Renner, Ke Zhang, Ruijia Cheng, Wenjuan Zhang, Joel Tetreault, Alejandro Jaimes-Larrarte
| Challenge: | Automated summarization methods are efficient but can suffer from low quality. |
| Approach: | They conducted an experiment with 72 participants to compare post-editing provided summaries with manual summarization for summary quality, human efficiency, and user experience. |
| Outcome: | The results show that post-editing improves summary quality, human efficiency, and user experience on formal (XSum news) and informal (Reddit posts) text. |
SummerTime: Text Summarization Toolkit for Non-experts (2021.emnlp-demo)
Copied to clipboard
Ansong Ni, Zhangir Azerbayev, Mutethia Mutuma, Troy Feng, Yusen Zhang, Tao Yu, Ahmed Hassan Awadallah, Dragomir Radev
| Challenge: | Recent advances in summarization provide models that can generate high quality summaries . a new toolkit for summarizing text is being developed to make it easier for non-experts to keep track of them. |
| Approach: | They develop a toolkit for text summarization that integrates with libraries designed for NLP researchers. |
| Outcome: | SummerTime is a toolkit for text summarization, including models, datasets, and evaluation metrics. |
PunKtuator: A Multilingual Punctuation Restoration System for Spoken and Written Text (2021.eacl-demos)
Copied to clipboard
| Challenge: | Prior punctuation restoration methods have focused on using lexical features, prosodic features or combination of both. |
| Approach: | They propose a multitask modeling approach to restore punctuation in multiple high resource languages using acoustic models and a computational model. |
| Outcome: | The proposed system can restore punctuation in Germanic, Romanic and low resource languages without extensive knowledge of grammar or syntax. |
SummVD : An efficient approach for unsupervised topic-based text summarization (2022.aacl-main)
Copied to clipboard
| Challenge: | SummVD is an unsupervised extractive summarization method that uses word clustering to reduce word embeddings. |
| Approach: | They propose a method for automatic unsupervised extractive summarization using word clustering and singular value decomposition. |
| Outcome: | The proposed method outperforms other extractive methods using several corpora of different nature. |
VizSeq: a visual analysis toolkit for text generation tasks (D19-3)
Copied to clipboard
| Challenge: | Several softwares for text evaluation are available that do not provide detailed examples. |
| Approach: | They propose a visual analysis toolkit for instance-level and corpus-level system evaluation on a wide variety of text generation tasks. |
| Outcome: | The proposed toolkit covers most common n-gram metrics and latest embedding-based metrics such as BERTScore. |
Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing text infilling objectives for pretrained language models require self-supervision by masking out tokens or spans in text. |
| Approach: | They propose to extend text infilling to a self-supervised sequence-to-sequence (Seq2Sequen) task. |
| Outcome: | The proposed task improves the model's performance on various natural language generation tasks. |
SummHelper: Collaborative Human-Computer Summarization (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Existing approaches for text summarization are mostly automated, with limited space for human intervention and control. |
| Approach: | They propose a 2-phase summarization assistant that facilitates human-machine collaboration . it suggests possible content and generates a coherent summary from these selections . authors hope to improve the efficiency of the computer and human-involved approach . |
| Outcome: | The proposed summarization assistant is a 2-phase summarizing assistant . it suggests potential content and consolidates the output with visual mappings . the proposed system is available for free on youtube . |
FineSurE: Fine-grained Summarization Evaluation using LLMs (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for text summarization evaluation do not correlate well with human judgments . evaluators that use Likert scale scores are limited in their ability to perform deeper analysis. |
| Approach: | They propose a fine-grained evaluator specifically tailored for the summarization task using large language models. |
| Outcome: | The proposed method improves on open-source and proprietary LLMs and shows better completeness and conciseness than existing methods. |
A Cascade Approach to Neural Abstractive Summarization with Content Selection and Fusion (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing systems that perform content selection and surface realization are not able to provide sufficient training data for news summarization. |
| Approach: | They propose to use a cascade architecture to perform content selection and surface realization together to generate abstracts. |
| Outcome: | The proposed architecture outperforms or outranks existing systems in terms of content selection and surface realization. |
Echoes from Alexandria: A Large Resource for Multilingual Book Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent research in text summarization has focused on news stories, where texts are typically short and have strong layout features. |
| Approach: | They propose a resource for multilingual book summarization that uses a new extractive-then-abstractive baseline to compare the results. |
| Outcome: | The proposed resource is the largest and first to be multilingual, featuring 5 languages and 25 language pairs. |
Identifying the Most Dominant Event in a News Article by Mining Event Coreference Relations (N18-2)
Copied to clipboard
| Challenge: | Identifying the most dominant and central event of a document is useful for many applications, says a new study . identifying the most prominent event in a news article is useful in text summarization, storyline generation and text segmentation. |
| Approach: | They propose to detect the most dominant and central event of a document . central event usually has many coreferential event mentions scattered throughout document a . |
| Outcome: | The proposed task can detect the most dominant and central event in a document . it can be used for text summarization, storyline generation and text segmentation . |
Noisy Self-Knowledge Distillation for Text Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to summarize text using a single reference and noisy datasets are ill-suited to summarising on single reference datasets. |
| Approach: | They propose to use self-knowledge distillation to improve text summarization by generating smoothed labels for students and teachers to reduce model uncertainty. |
| Outcome: | The proposed framework improves on pretrained and non-pretrained models on three benchmarks. |
A Corpus to Learn Refer-to-as Relations for Nominals (L18-1)
Copied to clipboard
| Challenge: | Existing work on how to learn refer-to-as relations from large unlabeled corpora lacks coreferential information. |
| Approach: | They propose to use Wikipedia to generate coreferential neural embeddings for nominals . they use coreference resolution as a proxy to evaluate the neural embeds for noun phrases . |
| Outcome: | The proposed dataset can be leveraged to construct representations for coreferential nominals from Wikipedia. |
Compositional Zero-Shot Domain Transfer with Text-to-Text Models (2023.tacl-1)
Copied to clipboard
Fangyu Liu, Qianchu Liu, Shruthi Bannur, Fernando Pérez-García, Naoto Usuyama, Sheng Zhang, Tristan Naumann, Aditya Nori, Hoifung Poon, Javier Alvarez-Valle, Ozan Oktay, Stephanie L. Hyland
| Challenge: | Existing approaches to zero-shot domain transfer are limited by domain gap and lack of in-domain labels. |
| Approach: | They propose a compositional transfer learning framework (DoT51) that learns domain knowledge and task knowledge in a multi-task manner without access to in-domain labels. |
| Outcome: | The proposed framework outperforms the current state-of-the-art in zero-shot domain transfer by over 7 absolute points in accuracy on RadNLI. |
MLASK: Multimodal Summarization of Video-based News Articles (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies on multimodal summarization have shown that the benefits of pre-training and using additional modalities in the input are not orthogonal. |
| Approach: | They propose to use a dataset to train a multimodal article summarization model by automatically crawling several news websites. |
| Outcome: | The proposed dataset can be used to model multimodal summarization by training a Transformer-based neural model. |
Controllable Summarization with Constrained Markov Decision Process (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing controllable summarization models do not allow users to specify their preference for a particular attribute of the generated summaries. |
| Approach: | They propose a novel training framework based on Constrained Markov Decision Process (CMDP) that includes a reward function and constraints to facilitate better summarization control. |
| Outcome: | The proposed model can be applied to control important attributes of summarization, including length, covered entities, and abstractiveness, while complying with a given attribute’s requirement. |
Hengam: An Adversarially Trained Transformer for Persian Temporal Tagging (2022.aacl-main)
Copied to clipboard
| Challenge: | A wide array of natural language processing (NLP) applications relies on accurately identifying events and their respective occurrence times. |
| Approach: | They propose an adversarially trained transformer for Persian temporal tagging that can generalize over the HengamTagger’s rules. |
| Outcome: | The proposed tool outperforms state-of-the-art methods on a diverse and manually created dataset. |
TACO-RL: Task Aware Prompt Compression Optimization with Reinforcement Learning (2025.findings-acl)
Copied to clipboard
Shivam Shandilya, Menglin Xia, Supriyo Ghosh, Huiqiang Jiang, Jue Zhang, Qianhui Wu, Victor Rühle, Saravan Rajmohan
| Challenge: | Existing prompt compression techniques rely on sub-optimal metrics such as information entropy or model it as a task-agnostic token classification problem that fails to capture task-specific information. |
| Approach: | They propose a task-aware prompt compression method that leverages existing Transformer encoders and a lightweight REINFORCE algorithm to ensure low latency requirements. |
| Outcome: | The proposed method improves task performance by 8% - 189% on three diverse and challenging tasks over state-of-the-art techniques while satisfying the same compression rate and latency requirements. |
A Graph-theoretic Summary Evaluation for ROUGE (D18-1)
Copied to clipboard
| Challenge: | ROUGE is one of the most widely used evaluation metrics for text summarization. |
| Approach: | They propose to use ROUGE to evaluate summaries based on lexical and semantic similarities. |
| Outcome: | The proposed method improves ROUGE's correlation with human judgments by exploiting lexical and semantic similarities. |
PrahokBART: A Pre-trained Sequence-to-Sequence Model for Khmer Natural Language Generation (2025.coling-main)
Copied to clipboard
| Challenge: | Pre-trained sequence-to-sequence models are typically pretrained on extensive raw text corpora and fine-tuned on task-specific data. |
| Approach: | They introduce a pre-trained sequence-to-sequence model trained from scratch for Khmer using carefully curated Khmer and English corpora. |
| Outcome: | The proposed model outperforms existing models on three generative tasks and is data-efficient and effective in enhancing performance across various natural language generation tasks. |
Prompt Perturbation Consistency Learning for Robust Language Models (2024.findings-eacl)
Copied to clipboard
Yao Qiang, Subhrangshu Nandi, Ninareh Mehrabi, Greg Ver Steeg, Anoop Kumar, Anna Rumshisky, Aram Galstyan
| Challenge: | Large language models have demonstrated impressive performance on a number of natural language processing tasks, such as question answering and text summarization. |
| Approach: | They propose a method to reduce the performance drop of large language models by regularizing the divergence between losses from clean and perturbed samples. |
| Outcome: | The proposed approach recovers on average 59% and 69% of the performance drop for IC and SF tasks while using ten times fewer augmented data samples. |
Efficient Out-of-Domain Detection for Sequence to Sequence Models (2023.findings-acl)
Copied to clipboard
Artem Vazhentsev, Akim Tsvigun, Roman Vashurin, Sergey Petrakov, Daniil Vasilev, Maxim Panov, Alexander Panchenko, Artem Shelmanov
| Challenge: | Sequence-to-sequence (seq2sequ) models are a ubiquitous tool for text generation but they are not suitable for many other tasks. |
| Approach: | They propose to use UE techniques to identify out-of-domain (OOD) inputs where the model is susceptible to errors. |
| Outcome: | The proposed methods outperform heavyweight ensembles on the task of OOD detection. |
Improving Abstractive Dialogue Summarization with Hierarchical Pretraining and Topic Segment (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for meeting summary have limited the ability to deal with long-term dependency. |
| Approach: | They propose a hierarchical transformer encoder-decoder network with multi-task pre-training to capture key sentences at word level and generate them at word-level. |
| Outcome: | The proposed model is superior to the previous methods in meeting summary datasets AMI and ICSI. |
Searching for Effective Neural Extractive Summarization: What Works and What’s Next (P19-1)
Copied to clipboard
| Challenge: | Recent years have seen success in the use of deep neural networks on text summarization, but there is no clear understanding of why they perform so well or how they might be improved. |
| Approach: | They propose to use different types of model architectures to improve extractive summarization systems. |
| Outcome: | The proposed framework achieves state-of-the-art on CNN/DailyMail by a large margin based on observations and analysis. |
Spurious Correlations in Reference-Free Evaluation of Text Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work suggests that reference-free evaluation metrics may rely on spurious correlations with human judgments. |
| Approach: | They propose to use model-based, reference-free evaluation metrics to evaluate natural language generation systems. |
| Outcome: | The proposed metrics achieve high correlations with human judgments, but they may not be robust enough to evaluate their efficacy and robustness. |
Improving Absent Keyphrase Generation with Diversity Heads (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to generating keyphrases for a given text are limited to extracting only the keyphrase that is directly seen in the document. |
| Approach: | They propose to treat present keyphrase extraction as a sequence labeling problem and treat absent keyphrases together in a text-to-text generation framework during training. |
| Outcome: | The proposed model improves on the state-of-the-art for present keyphrase extraction and five datasets for absent keyphrase generation among the six English datasets. |
Weakly supervised discourse segmentation for multiparty oral conversations (2021.emnlp-main)
Copied to clipboard
| Challenge: | Discourse segmentation is the first step of discourse analysis. |
| Approach: | They propose a weak supervision approach to adapt a latent model to French conversation transcripts with a linguistic and acoustic input. |
| Outcome: | The proposed model improves in situations where speaker turns are lacking or noisy, gaining up to 13% in F-score. |
M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitions (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for retrieving relevant memories from an external database are coarse-grained and can cause noise and focus on crucial memories. |
| Approach: | They propose a multiple partition paradigm for RAG where each database partition serves as a basic unit for execution. |
| Outcome: | The proposed framework outperforms baseline methods on three language generation tasks on seven datasets. |
RefSum: Refactoring Neural Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for text summarization are limited by limitations of reranking or stacking. |
| Approach: | They propose a framework that provides a unified view of text summarization and summaries combination. |
| Outcome: | The proposed method can be used by researchers as an off-the-shelf tool to achieve further performance improvements. |
Constructing contrastive samples via summarization for text classification with limited annotations (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Various contrastive learning methods have been developed and lead to state-of-the-art performance in many computer vision tasks. |
| Approach: | They propose a method to construct efficient contrastive samples using text summarization to gain better representations of text classification tasks with limited annotations. |
| Outcome: | The proposed framework gains better representations on text classification tasks with limited annotations and is compared with existing methods on real-world text classification datasets. |
EDU-level Extractive Summarization with Varying Summary Lengths (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies on extractive summarization use finer-grained elementary discourse units . few studies exploited finer grained EDUs with little analysis and justification for the extractive unit selection . |
| Approach: | They propose an extractive model with Varying summary lengths that extracts fixed top-k salient sentences from the document as a summary. |
| Outcome: | The proposed model performs better on ROUGE scores than state-of-the-art models. |
Towards a Unified Multi-Dimensional Evaluator for Text Generation (2022.emnlp-main)
Copied to clipboard
Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, Jiawei Han
| Challenge: | Existing evaluation frameworks for natural language generation are dominated by similarity-based metrics. |
| Approach: | They propose a multi-dimensional evaluator for natural language generation that integrates multiple dimensions into one evaluer. |
| Outcome: | The proposed evaluator improves on three typical NLG tasks and improves with external knowledge. |
Conditional Neural Generation using Sub-Aspect Functions for Extractive News Summarization (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in text summarization have overcome position bias in news articles . however, there are long-standing, unresolved challenges in extractive summarizing . |
| Approach: | They propose a neural framework that can flexibly control summary generation by introducing a set of sub-aspect functions. |
| Outcome: | The proposed framework can flexibly control summary generation by introducing sub-aspect functions . extracted summaries with minimal position bias are comparable with standard models . |
FIDELITY: Fine-grained Interpretable Distillation for Effective Language Insights and Topic Yielding (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for topic modeling generate contextually specific and semantically intuitive topics, especially in dynamic environments and low-resource languages. |
| Approach: | They propose a hybrid method that combines topic modeling and text summarization to produce fine-grained, semantically rich, and contextually relevant output. |
| Outcome: | FIDELITY outperforms traditional models in topic diversity, similarity, and ability to process new, unseen documents. |
Mask Attention Networks: Rethinking and Strengthen Transformer (2021.naacl-main)
Copied to clipboard
Zhihao Fan, Yeyun Gong, Dayiheng Liu, Zhongyu Wei, Siyuan Wang, Jian Jiao, Nan Duan, Ruofei Zhang, Xuanjing Huang
| Challenge: | Existing research explores to enhance the two sublayers separately to improve the capability of Transformer for text representation. |
| Approach: | They propose to combine SAN and Feed-Forward Networks to create a dynamic mask attention network with a learnable mask matrix which can model localness adaptively. |
| Outcome: | The proposed model outperforms the original Transformer on translation and text summarization tasks. |
Evolutionary Guided Decoding: Iterative Value Refinement for LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for directing language model outputs are limited in their accuracy due to a distributional gap . existing methods train static value functions on trajectories sampled exclusively from the base policy . |
| Approach: | They propose a framework to bridge a distributional gap in the accuracy of value functions . they propose RLHF to align language models with human values and task requirements . |
| Outcome: | The proposed framework reduces computational costs and improves value function accuracy by leveraging principled value function optimization. |
G-Eval: NLG Evaluation using Gpt-4 with Better Human Alignment (2023.emnlp-main)
Copied to clipboard
| Challenge: | Conventional reference-based metrics have low correlation with human judgments, especially for open-ended generation tasks. |
| Approach: | They propose to use large language models as reference-free NLG evaluators to assess the quality of NLG outputs. |
| Outcome: | The proposed framework outperforms all previous methods in two generation tasks, and has a Spearman correlation of 0.514 with human on summarization task, and a large variance in human judgments. |
TriSum: Learning Summarization Ability from Large Language Models with Structured Rationale (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have advanced tasks like text summarization, but their size and computational demands limit their use in resource-constrained and privacy-centric settings. |
| Approach: | They propose a framework for distilling LLMs’ text summarization abilities into a compact, local model using a curriculum learning strategy that evolves from simple to complex tasks. |
| Outcome: | The proposed framework outperforms baseline models on CNN/DailyMail, XSum, and ClinicalTrial, and improves interpretability by providing insights into the summarization rationale. |
An Investigation of Evaluation Methods in Automatic Medical Note Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that doctors can save significant amounts of time when using automatic note generation. |
| Approach: | They propose task-specific metrics for automatic note generation from medical conversation summarization and generation, including knowledge-graph embedding-based metrics, customized model-based measures with domain-specific weights, and ensemble metrics. |
| Outcome: | The proposed evaluation metrics are compared to existing models and can have different behaviors on different types of clinical notes datasets. |
Semantic-Preserving Abstractive Text Summarization with Siamese Generative Adversarial Net (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods focus on attention mechanism, but they are not suitable for abstractive text summarization. |
| Approach: | They propose a siamese generative adversarial net for abstractive text summarization which preserves the main semantics of the source text and the target summary. |
| Outcome: | The proposed model can preserve the main semantics of the source text and target summary. |
Mark-Evaluate: Assessing Language Generation using Population Estimation Methods (2020.coling-main)
Copied to clipboard
| Challenge: | Existing population estimation methods focus on open populations or closed populations, but our methods show a higher correlation to human evaluation than existing metrics on several challenging tasks. |
| Approach: | They propose a family of metrics to assess language generation derived from population estimation methods widely used in ecology. |
| Outcome: | The proposed methods show a higher correlation to human evaluation than existing metrics on several challenging tasks, namely unconditional language generation, machine translation, and text summarization. |
KLearn: Background Knowledge Inference from Summarization Data (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing summarization approaches focus on relevance, but background knowledge is not modeled by simple aggregation of large background corpora. |
| Approach: | They propose to define summary scoring functions that explicitly model background knowledge and compare them to baselines to see if they fit human judgments. |
| Outcome: | The proposed framework fits human judgments significantly better than baselines. |
MTG: A Benchmark Suite for Multilingual Text Generation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Using MTG, we train and evaluate multilingual text generation models using human-annotated data. |
| Approach: | They propose a multilingual multiway text generation dataset with 400k human-annotated data that includes four generation tasks across five languages. |
| Outcome: | The proposed dataset includes four generation tasks across five languages (English, German, French, Spanish and Chinese) it provides comprehensive evaluations with diverse generation scenarios. |
SmartBench: Is Your LLM Truly a Good Chinese Smartphone Assistant? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks for Large Language Models focus on objective tasks like mathematics and coding in English, which do not reflect the practical use cases of on-device LLMs in real-world mobile scenarios. |
| Approach: | They propose a benchmark to evaluate the capabilities of on-device Large Language Models in Chinese mobile contexts. |
| Outcome: | The proposed framework evaluates on-device LLMs and MLLMs in Chinese . it provides a standardized framework for evaluating LLM performance on real smartphones . |
MReD: A Meta-Review Dataset for Structure-Controllable Text Generation (2022.findings-acl)
Copied to clipboard
| Challenge: | a new text generation dataset is needed to controllable text summarization, but it lacks the domain knowledge. |
| Approach: | They propose to use existing text generation datasets to leverage input and control signals . they propose to annotate each meta-review sentence manually with a control signal . |
| Outcome: | The proposed method can be used to control the structure of a text generation dataset . it can be applied to a variety of tasks, including a task with a large number of meta-review sentences . |
BenLLM-Eval: A Comprehensive Evaluation into the Potentials and Pitfalls of Large Language Models on Bengali NLP (2024.lrec-main)
Copied to clipboard
Mohsinul Kabir, Mohammed Saidul Islam, Md Tahmid Rahman Laskar, Mir Tafseer Nayeem, M Saiful Bari, Enamul Hoque
| Challenge: | Large Language Models (LLMs) have emerged as one of the most important breakthroughs in natural language processing. |
| Approach: | They propose to evaluate LLMs in Bengali to benchmark their performance . they select Bangla NLP tasks such as text summarization, question answering, paraphrasing . |
| Outcome: | The proposed model performs better in some tasks than current models, but in most tasks, it is poor . |
Beyond Semantic Entropy: Boosting LLM Uncertainty Quantification with Pairwise Semantic Similarity (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate long one-sentence responses that are less effective because they overlook two crucial factors: intra-cluster similarity and inter-c cluster similarity. |
| Approach: | They propose a method that generalizes semantic entropy and uses token probabilities to quantify uncertainty in large language models. |
| Outcome: | The proposed method can be extended to white-box settings by incorporating token probabilities. |
UMSE: Unified Multi-scenario Summarization Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Summarization quality evaluation is a non-trivial task in text summarization. |
| Approach: | They propose a unified multi-scenario summarization evaluation model that shares cross-sceenario knowledge and uses a self-supervised training paradigm to optimize the model without extra human labeling. |
| Outcome: | The proposed model can achieve comparable performance with existing methods for three evaluation scenarios. |
Enhancing Language Generation with Effective Checkpoints of Pre-trained Language Model (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to exploit PrLMs for NLG tasks do not get as much performance gain as in the NLU task. |
| Approach: | They propose a method to integrate public checkpoints of PrLMs for the most convenience. |
| Outcome: | The proposed method significantly improves the quality of the language generation tasks on 6 different kinds of PrLMs. |
Few-shot Query-Focused Summarization with Prefix-Merging (2022.emnlp-main)
Copied to clipboard
| Challenge: | Query-focused summarization has been considered as an important extension for text summarizing . lack of large-scale datasets hinders its development . |
| Approach: | They propose to integrate text summarization and question answering into a prefix-based pretraining strategy for few-shot learning in query-focused summarizing. |
| Outcome: | The proposed prefix-based pretraining outperforms fine-tuning on query-focused summarization. |
ZmBART: An Unsupervised Cross-lingual Transfer Framework for Language Generation (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in NLP focus on large annotated training data. |
| Approach: | They propose an unsupervised framework that does not use parallel or pseudo-parallel/back-translated data. |
| Outcome: | The proposed framework does not use parallel or pseudo-parallel/back-translated data. |
Read between the lines - Functionality Extraction From READMEs (2024.findings-naacl)
Copied to clipboard
| Challenge: | While text summarization is a well-known NLP task, functionality extraction from Git README files is based on a novel and useful variant of it. |
| Approach: | They propose a novel task called functionality extraction from Git README files that detects all the functionalities supported by the corresponding application software. |
| Outcome: | The proposed model outperforms baseline models that can be designed using popular black-box or white-box large language models (LLMs) such as ChatGPT and Bard. |
Does Pretraining for Summarization Require Knowledge Transfer? (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing theories claim that pretraining models learn linguistic knowledge from the pretraining corpus, but scientific explanations for these benefits remain unknown. |
| Approach: | They propose to use random character n-grams to test models on real corpora to see if the small residual benefit of using real data could be accounted for by the structure of the pretraining task. |
| Outcome: | The proposed task performs on documents consisting of character n-grams, whereas pretrained models perform on real corpora with no residual benefit. |
Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization (2024.findings-naacl)
Copied to clipboard
Yixin Liu, Alexander Fabbri, Jiawen Chen, Yilun Zhao, Simeng Han, Shafiq Joty, Pengfei Liu, Dragomir Radev, Chien-Sheng Wu, Arman Cohan
| Challenge: | Recent studies have found that large language models (LLMs) can achieve state-of-the-art performance on generic summarization benchmarks, but their performance on more complex summarizing task settings is less studied. |
| Approach: | They benchmark large language models on instruction controllable text summarization . they use 4 evaluation protocols and 11 LLMs to evaluate their performance . |
| Outcome: | The proposed model performs well on instruction controllable text summarization tasks with 4 evaluation protocols and 11 LLMs. |
From spoken dialogue to formal summary: An utterance rewriting for dialogue summarization (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing models focus more on the structure of summary, not on the personal and logical inconsistency problem. |
| Approach: | They propose a model to solve the problem of personal and logical inconsistency . they use an utterance rewriter to complete the ellipsis content of dialogue content . |
| Outcome: | The proposed model outperforms baseline models on both SAMSum and DialSum datasets. |
Developing the Bangla RST Discourse Treebank (L18-1)
Copied to clipboard
| Challenge: | a corpus in Bangla is annotated for coherence relations between text segments representing propositions . the corpus is a valuable resource for conducting discourse studies for Bangla . |
| Approach: | They propose to build a Bangla-annotated corpus which includes 266 Bangla texts . they use Rhetorical Structure Theory as the theoretical framework to develop the corpus . |
| Outcome: | The proposed corpus contains 266 Bangla texts annotated for coherence relations . the research could be used for discourse studies and for developing NLP applications . |
A Joint Model for Structure-based News Genre Classification with Application to Text Summarization (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models for structure-based news genre classification identify news structure types and news elements . authors show that the model outperforms variants that perform two tasks independently . |
| Approach: | They propose a joint model that identifies one of four commonly used news structures for a news article and recognizes a sequence of news elements within the article that define the corresponding news structure. |
| Outcome: | The proposed model outperforms variants that perform two tasks independently . it predicts news structure type and news elements and improves text summarization . |
Literature Retrieval for Precision Medicine with Neural Matching and Faceted Summarization (2020.findings-emnlp)
Copied to clipboard
| Challenge: | IR for precision medicine often involves looking for multiple pieces of evidence that characterize a patient case. |
| Approach: | They propose a document reranking approach that combines neural query-document matching and text summarization toward such retrieval scenarios. |
| Outcome: | The proposed approach achieves state-of-the-art performance on NIST's TREC-PM track dataset. |
Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization (2023.acl-long)
Copied to clipboard
| Challenge: | Contemporary leading-edge systems for abstractive (long) text summarization employ Transformer encoderdecoder architectures that only consider the nuclearity annotation . |
| Approach: | They propose to incorporate Rhetorical Structure Theory into a novel summarization model that incorporates both the types and uncertainty of rhetorical relations. |
| Outcome: | The proposed model outperforms state-of-the-art models on automatic metrics and human evaluation. |
NIPS Conversational Intelligence Challenge 2017 Winner System: Skill-based Conversational Agent with Supervised Dialog Manager (C18-1)
Copied to clipboard
| Challenge: | a dialog system developed for the 1st NIPS Conversational Intelligence Challenge 2017 (ConvAI) aims to train a bot capable of conversing with humans based on a given passage of text. |
| Approach: | They propose to implement a bot capable of conversing with humans based on a given passage of text. |
| Outcome: | The proposed bot won the 1st NIPS Conversational Intelligence Challenge 2017 (ConvAI) the system has been trained in a supervised setting using a dialogue manager . |
End-to-End AMR Coreference Resolution (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on AMR focuses on individual sentences, but there is a need for multi-sentence AMRs. |
| Approach: | They propose to use an end-to-end AMR coreference resolution model to generate multi-sentence AMRs. |
| Outcome: | The proposed model reduces error propagation and is more robust for both in- and out-domain situations. |
Analyzing and Evaluating Faithfulness in Dialogue Summarization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on faithfulness of text summarization have not been conducted on abstractive summarizing. |
| Approach: | They propose a method to evaluate faithfulness of dialogue summarization models by multi-choice questions. |
| Outcome: | The proposed method can facilitate the development of dialogue summarization systems. |
On the Multilingual Capabilities of Very Large-Scale English Language Models (2022.lrec-1)
Copied to clipboard
| Challenge: | Generative Pre-trained Transformers (GPTs) have been scaled to unprecedented sizes in the history of machine learning. |
| Approach: | They investigate the potential and limits of Generative Pre-trained Transformers in three tasks . they find it can be almost as useful for many languages as it is for English . |
| Outcome: | The proposed model can perform tasks in five different languages, and its potential is explored . it can learn from a few examples "via text interaction" and is scalable to many languages . |
CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluation methods for text summarization systems are limited to in-domain setting, where supervised pre-trained models are evaluated on the same dataset. |
| Approach: | They propose to use a cross-dataset evaluation approach to evaluate different summarization systems in a multi-domain setting. |
| Outcome: | The proposed model can be used to evaluate text summarization systems on different datasets. |
Event Graph based Sentence Fusion (2021.emnlp-main)
Copied to clipboard
| Challenge: | Sentence fusion is a conditional generation task that merges related sentences into a coherent text. |
| Approach: | They propose to build an event graph from the input sentences to capture related events in a structured way and use the constructed event graph to guide sentence fusion. |
| Outcome: | The proposed method achieves state-of-the-art on two datasets . it is based on the input sentences and shows that it is effective . |
Text Segmentation by Cross Segment Attention (2020.emnlp-main)
Copied to clipboard
| Challenge: | Document and discourse segmentation are two fundamental NLP tasks pertaining to breaking up text into constituents. |
| Approach: | They propose three transformer-based NLP models that break up text into constituents and compare them to previous approaches. |
| Outcome: | The proposed architectures reduce errors by a large margin on three datasets and improve performance on real-world datasets. |
Text Summarization with Pretrained Encoders (D19-1)
Copied to clipboard
| Challenge: | Existing pretraining languages such as ELMo and GPT have advanced a wide range of tasks. |
| Approach: | They propose a novel document-level encoder based on BERT which can express the semantics of a document and obtain representations for its sentences. |
| Outcome: | The proposed model achieves state-of-the-art in extractive and abstractive settings. |
Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation suffers from exposure bias and error propagation problem. |
| Approach: | They conduct a series of analyses to deeply understand the accuracy drop problem . they find that the left part of the translated sentence is often better than its right part . |
| Outcome: | The results show that the left part of the translated sentence is often better than its right part in left-to-right decoding models. |
Jointly Extracting and Compressing Documents with Summary State Representations (N19-1)
Copied to clipboard
| Challenge: | Text summarization is an important NLP problem with a wide range of applications in data-driven industries. |
| Approach: | They propose a neural model that extracts sentences from a document and compresses them. |
| Outcome: | The proposed model generates concise and informa-tive summaries on CNN/DailyMail and Newsroom datasets and human evaluations show it outperforms existing methods. |
Interpretable Automatic Fine-grained Inconsistency Detection in Text Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing factual consistency evaluation approaches for text summarization provide binary predictions and limited insights into the weakness of summarizing systems. |
| Approach: | They propose a task of fine-grained inconsistency detection to detect inconsistent summaries . they represent facts with semantic frames extracted by semantic role labeling and highlight related semantic frames to predict inconsistance . |
| Outcome: | The proposed model outperforms baseline models and provides evidence to support or refute the summary. |
Correction of Errors in Preference Ratings from Automated Metrics for Text Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation methods are over-confident in assigning significant differences between systems . Currently, the most reliable evaluation methods for text generation are human-based evaluations. |
| Approach: | They propose to combine human ratings with automated ratings to reduce the amount of human ratings needed to arrive at robust results. |
| Outcome: | The proposed evaluation protocol reduces the amount of human ratings by 50% while yielding the same evaluation outcome as the pure human evaluation in 95% of cases. |
CaLcs: Continuously Approximating Longest Common Subsequence for Sequence Level Optimization (D18-1)
Copied to clipboard
| Challenge: | Maximum-likelihood estimation (MLE) is widely used for text-generation based natural language processing applications. |
| Approach: | They propose a method to train models with maximum-likelihood estimation using a differentiable surrogate of longest common subsequence measure that captures sequence-level structure similarity. |
| Outcome: | Experimental results show that the proposed approach improves on the current MLE approach for downstream tasks like text summarization and machine translation. |
Leveraging Locality in Abstractive Text Summarization (2022.emnlp-main)
Copied to clipboard
Yixin Liu, Ansong Ni, Linyong Nan, Budhaditya Deb, Chenguang Zhu, Ahmed Hassan Awadallah, Dragomir Radev
| Challenge: | Neural attention models have improved on many natural language processing tasks, but their quadratic memory complexity hinders their applications in long text summarization. |
| Approach: | They propose to use a restricted context to study locality in text summarization . they propose to employ a quadratic memory growth with respect to the input length . |
| Outcome: | The proposed model has better performance than baseline models with efficient attention modules. |
Responsible AI Considerations in Text Summarization Research: A Review of Current Practices (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study examines research and reporting practices for text summarization tasks . text summaries are often overlooked by the responsible AI community . |
| Approach: | They examine research and reporting practices in the context of text summarization . they find that relatively few papers engage with possible stakeholders . |
| Outcome: | The findings highlight current research practices and provide recommendations on research directions. |
Abstractive Text-Image Summarization Using Multi-Modal Attentional Hierarchical RNN (D18-1)
Copied to clipboard
| Challenge: | Recent research shows the strength of the Encoder-Decoder model in text summarization. |
| Approach: | They propose to use the attentional hierarchical Encoder-Decoder model to summarize a text document and its accompanying images simultaneously and then to align the sentences and images in summaries. |
| Outcome: | The proposed model outperforms the existing methods that do not consider images . it can generate informative summaries of images, and it can be used to summarize documents . |
Prompt Chaining or Stepwise Prompt? Refinement in Text Summarization (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) can improve summary quality by mirroring a human-like iterative process of critique and refinement starting from the initial draft. |
| Approach: | They propose to use Prompt Chaining and Stepwise Prompting to perform iterative refinement . they propose to combine the two methods to produce a more favorable outcome . |
| Outcome: | The proposed methods can improve summary quality by mirroring a human-like iterative process . the results show that the prompt chaining method produces a more favorable outcome . |
Revisiting Grammatical Error Correction Evaluation and Beyond (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pretraining-based (PT) evaluation metrics are not effective for training grammatical error correction systems. |
| Approach: | They propose a pretraining-based GEC evaluation metric which only uses PT-based metrics to score the corrected parts of the system. |
| Outcome: | The proposed evaluation metric outperforms existing methods on a CoNLL14 evaluation task. |
POS-Constrained Parallel Decoding for Non-autoregressive Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing non-autoregressive generation systems face multimodality problem due to conditionally independent decoding. |
| Approach: | They propose to incorporate linguistic structure into NAG inference instead of teacher AG . they propose a method that provides a specific POS sequence to constrain the NAG model . |
| Outcome: | The proposed method improves NAG models on four text generation tasks to a greater extent compared to knowledge distillation. |
Enhancing Extractive Text Summarization with Topic-Aware Graph Neural Networks (2020.coling-main)
Copied to clipboard
| Challenge: | Existing extractive summarization models hardly capture inter-sentence relationships, especially in long documents. |
| Approach: | They propose to use a graph neural network to capture inter-sentence relationships efficiently via graph-structured document representation. |
| Outcome: | The proposed model outperforms existing models on CNN/DM and NYT datasets and significantly outperfies them on longer documents. |
On Learning to Summarize with Large Language Models as References (2024.naacl-long)
Copied to clipboard
Yixin Liu, Kejian Shi, Katherine He, Longtian Ye, Alexander Fabbri, Pengfei Liu, Dragomir Radev, Arman Cohan
| Challenge: | Recent studies have found that summaries generated by large language models (LLMs) are favored by human annotators when compared to reference summary from widely used summarization datasets. |
| Approach: | They propose to use large language models (LLMs) as reference learning settings for smaller text summarization models to investigate whether their performance can be substantially improved. |
| Outcome: | The proposed model outperforms standard supervised fine-tuning and human evaluations while retaining human-level performance. |
Hallucination Diversity-Aware Active Learning for Text Summarization (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for alleviating hallucinations require costly human annotations . Existing approaches focus on a specific type of hallucinism, which limits their effectiveness . |
| Approach: | They propose a method to detect hallucinations from errors in semantic frame, discourse and content verifiability in LLM summarization using HAllucination Diversity-Aware Sampling. |
| Outcome: | The proposed framework reduces the need for costly human annotations to correct hallucinations in LLM outputs. |
Reference and Document Aware Semantic Evaluation Methods for Korean Language Summarization (2020.coling-main)
Copied to clipboard
Dongyub Lee, Myeong Cheol Shin, Taesun Whang, Seungwoo Cho, Byeongil Ko, Daniel Lee, EungGyun Kim, Jaechoon Jo
| Challenge: | Existing methods for text summarization are based on recall-oriented understudy for gisting evaluation (ROUGE) scores do not reflect semantic meaning correspondences between generated and reference summaries. |
| Approach: | They propose to use Korean as a summarization language to generate a shorter form of text from the source document preserving salient information. |
| Outcome: | The proposed evaluation metrics improve the correlation between the metrics and human judgment. |
Metrics also Disagree in the Low Scoring Range: Revisiting Summarization Evaluation Metrics (2020.coling-main)
Copied to clipboard
| Challenge: | In text summarization evaluation, evaluating the efficacy of automated metrics without human judgments has become popular. |
| Approach: | They revisit their experiments and find that automatic metrics disagree when ranking high-scoring summaries. |
| Outcome: | The proposed method is a human judgment-free method, but it is not a meta-evaluation strategy. |
Hyperparameter-Free Approach for Faster Minimum Bayes Risk Decoding (2024.findings-acl)
Copied to clipboard
| Challenge: | Minimum Bayes-Risk (MBR) decoding is a powerful alternative to beam search for text generation tasks. |
| Approach: | They propose a method that uses a hyperparameter-free method to run MBR decoding efficiently. |
| Outcome: | The proposed method reduces the amount of computation but requires hyperparameter tuning to be effective. |
Selective Token Generation for Few-shot Natural Language Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Experimental results show that the proposed selective token generation algorithm outperforms the previous additive learning algorithms based on the PLMs. |
| Approach: | They propose an additive learning algorithm that selectively outputs language tokens between a task-general PLM and a specific adapter during training and inference. |
| Outcome: | The proposed algorithm outperforms existing methods on few-shot natural language generation tasks. |
Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent research shows that large language models (LLMs) perform poorly at segment level. |
| Approach: | They propose a new prompting method that emulates the commonly accepted human evaluation framework . they will release their code and scripts to facilitate the community . |
| Outcome: | The proposed method is based on the human evaluation framework MQM and produces explainable and reliable MT evaluations at both the system and segment level. |
Text-Tuple-Table: Towards Information Integration in Text-to-Table Generation via Global Tuple Extraction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to condensing textual information into concise and structured tables are limited in their applicability in broader contexts. |
| Approach: | They propose a benchmark dataset for generating summary tables of competitions based on real-time commentary texts that incorporates large-scale textual information into concise and structured tables. |
| Outcome: | The proposed method exhibits strong generalization abilities, surpassing previous approaches on several other text-to-table datasets. |
Multi-Dimensional Evaluation of Text Summarization with In-Context Learning (2023.findings-acl)
Copied to clipboard
Sameer Jain, Vaishakh Keshava, Swarnashree Mysore Sathyendra, Patrick Fernandes, Pengfei Liu, Graham Neubig, Chunting Zhou
| Challenge: | In-context learning-based evaluators are competitive with learned evaluation frameworks for text summarization tasks. |
| Approach: | They propose to use large language models as multi-dimensional evaluators using in-context learning to evaluate text summarization tasks. |
| Outcome: | The proposed frameworks are competitive with existing frameworks on relevance and factual consistency, the authors show . |
DecompEval: Evaluating Generated Texts as Unsupervised Decomposed Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation metrics for natural language generation (NLG) tasks face the challenges on generalization ability and interpretability. |
| Approach: | They propose a metric that evaluates natural language generation tasks as an instruction-style question answering task and utilizes instruction-tuned pre-trained language models without training on evaluation datasets. |
| Outcome: | The proposed metric achieves state-of-the-art performance in untrained metrics for evaluating text summarization and dialogue generation, which exhibits strong dimension-level / task-level generalization ability and interpretability. |
GRETEL: Graph Contrastive Topic Enhanced Language Model for Long Document Extractive Summarization (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to capture and integrate global semantic information are limited due to their limited ability to capture long-range dependencies. |
| Approach: | They propose a graph contrastive topic enhanced language model that integrates a neural topic model with a pre-trained language model to capture global contextual semantics. |
| Outcome: | The proposed model outperforms existing methods on general domain and biomedical datasets. |
LaMDA: Large Model Fine-Tuning via Spectrally Decomposed Low-Dimensional Adaptation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Low-rank adaptation (LoRA) fine-tunes large language models due to its significant reduction in trainable parameters, but its backward updates require storing high-dimensional intermediate activations and optimizer states, requiring high peak GPU memory. |
| Approach: | They propose a low-dimensional adaptation approach to fine-tune large language models which freezes a first projection matrix while introducing a lower-dimensional trainable square matrix. |
| Outcome: | The proposed approach reduces trainable parameters and peak GPU memory footprint while preserving low-dimensional trainable square matrix. |
Improving Abstractive Dialogue Summarization with Speaker-Aware Supervised Contrastive Learning (2022.coling-1)
Copied to clipboard
| Challenge: | Existing summarization systems based on pre-trained models cannot recognize the unique format of the speaker-utterance pair well in the dialogue. |
| Approach: | They propose three speaker-aware supervised contrastive learning tasks to solve the speaker identification problem in dialogue summarization task. |
| Outcome: | The proposed methods improve on two mainstream dialogue summarization datasets. |
Prompted Opinion Summarization with GPT-3.5 (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent years have seen several shifts in summarization research, including extractive models. |
| Approach: | They propose a pipeline method for applying GPT-3.5 to summarize user reviews . they propose three new metrics targeting faithfulness, factuality, and genericity . |
| Outcome: | The proposed methods perform well in opinion summarization, the authors show . they also show that standard evaluation metrics do not reflect this performance . |
Effective Sequence-to-Sequence Dialogue State Tracking (2021.emnlp-main)
Copied to clipboard
| Challenge: | Using Sequence-to-Sequence models for dialogue state tracking remains an understudied topic. |
| Approach: | They propose to use a pre-training objective and a dialogue context representation to investigate this problem. |
| Outcome: | The proposed model is more effective than auto-regressive language modeling, the authors show . the proposed model may have a hard time recovering from earlier mistakes, they say . |
Facilitating Cognitive Accessibility with LLMs: A Multi-Task Approach to Easy-to-Read Text Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to make complex texts more accessible for people with cognitive impairments are time-consuming and resource-intensive. |
| Approach: | They propose a multi-task learning approach that trains models jointly on text summarization, text simplification, and ETR generation. |
| Outcome: | The proposed approach outperforms other approaches in in-domain settings while achieving better generalization in out-of-domain scenarios. |
Compression, Transduction, and Creation: A Unified Framework for Evaluating Natural Language Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Natural language generation (NLG) tasks have complex nature and require manual evaluation. |
| Approach: | They propose a unifying perspective based on the nature of information change in NLG tasks . they propose 'information alignment' metrics that can be used to evaluate different aspects of NLG . |
| Outcome: | The proposed metrics achieve stronger or comparable correlations with human judgement compared to state-of-the-art metrics in diverse tasks. |
Contrastive Error Attribution for Finetuned Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for error tracing do not detect faithfulness errors in NLG datasets. |
| Approach: | They propose a framework to identify and remove low-quality training instances that lead to undesirable outputs. |
| Outcome: | The proposed method outperforms existing methods for detecting faithfulness errors in NLG datasets. |
Improving Factual Consistency of News Summarization by Contrastive Preference Optimization (2024.findings-emnlp)
Copied to clipboard
Huawen Feng, Yan Fan, Xiong Liu, Ting-En Lin, Zekun Yao, Yuchuan Wu, Fei Huang, Yongbin Li, Qianli Ma
| Challenge: | Recent advances in news summarization have created problems with “hallucinations” that are factually inconsistent with the source text. |
| Approach: | They propose to disentangle LLMs’ propensities to generate faithful and fake content by adopting a probing-based specific training method to improve their capacity of distinguishing two types of propensity. |
| Outcome: | The proposed method disentangles LLMs’ propensities to generate faithful and fake content and improves their ability to distinguish between two types of propensity. |
ModeX: Evaluator-Free Best-of-N Selection for Open-Ended Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to selecting a single high-quality output from multiple generations are limited in their applicability and efficiency. |
| Approach: | They propose a method that generalizes majority voting to open-ended text generation . modeX leverages structural information across multiple generation paths to select a "modal" output . |
| Outcome: | The proposed framework outperforms standard single- and multi-path baselines in open-ended tasks. |
LightFormer: Light-weight Transformer Using SVD-based Weight Transfer and Parameter Sharing (2023.findings-acl)
Copied to clipboard
| Challenge: | Deploying Transformer networks on resource-constrained edge devices is challenging. |
| Approach: | They propose a low-rank factorization initialized by SVD-based weight transfer and parameter sharing to compress and accelerate Transformer networks. |
| Outcome: | The proposed method achieves similar performance to the baseline Transformer with 3.8 times and 1.8 times fewer parameters and achieves 2.3 times speedup and 1.5 times speed up respectively. |
FastMem: Fast Memorization of Prompt Improves Context Awareness of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models struggle with context awareness, leading to inaccuracies in tasks requiring faithful adherence to provided information. |
| Approach: | They propose a method to enhance LLMs' context awareness by updating only the last Feed-Forward Network module to maximize the likelihood of the prompt before inference . |
| Outcome: | The proposed method improves the accuracy of Llama 3-8B-Inst on the NQ-SWAP dataset from 59.1% to 71.6% and reduces the output structure failure rate of Qwen 1.5-4B-Chat from 34.9% to 25.5%. |
Towards Multi-dimensional Evaluation of LLM Summarization across Domains and Languages (2025.acl-long)
Copied to clipboard
Hyangsuk Min, Yuho Lee, Minjeong Ban, Jiaqi Deng, Nicole Hee-Yeon Kim, Taewon Yun, Hang Su, Jason Cai, Hwanjun Song
| Challenge: | Existing evaluation frameworks for text summarization lack domain-specific assessment criteria and are predominantly English-centric. |
| Approach: | They propose a multi-dimensional, multi-domain evaluation of summarization in English and Chinese that incorporates specialized assessment criteria for each domain and leverages a debate system to enhance annotation quality. |
| Outcome: | The proposed evaluation framework provides a multi-dimensional, multi-domain evaluation of summarization in English and Chinese. |
Distilling Knowledge Learned in BERT for Text Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models such as BERT have revolutionized the state of the art in many language understanding tasks. |
| Approach: | They propose a conditional masked language modeling approach to fine tune BERT on target generation tasks by imposing global sequence-level supervision on conventional Seq2Seq models. |
| Outcome: | The proposed model outperforms strong Transformer baselines on multiple language generation tasks such as machine translation and text summarization. |
Detecting Errors through Ensembling Prompts (DEEP): An End-to-End LLM Framework for Detecting Factual Errors (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for detecting factual errors in text summarization are inadequate for the task. |
| Approach: | They propose an end-to-end large language model framework for detecting factual errors in text summarization. |
| Outcome: | The proposed framework achieves state-of-the-art (SOTA) balanced accuracy on the AggreFact-XSUM FTSOTA, TofuEval Summary-Level, and HaluEVAL Summarization benchmarks. |
Improving Text Generation with Student-Forcing Optimal Transport (2020.emnlp-main)
Copied to clipboard
Jianqiao Li, Chunyuan Li, Guoyin Wang, Hao Fu, Yuhchen Lin, Liqun Chen, Yizhe Zhang, Chenyang Tao, Ruiyi Zhang, Wenlin Wang, Dinghan Shen, Qian Yang, Lawrence Carin
| Challenge: | Maximum likelihood estimation (MLE) is used to train models, but during testing, the model is conditioned on previously generated tokens, resulting in exposure bias. |
| Approach: | They propose to use optimal transport to match the sequences generated in MLE and test modes to reduce exposure bias. |
| Outcome: | The proposed method is validated on machine translation, text summarization, and text generation tasks. |
Re-evaluating Evaluation in Text Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Automated evaluation metrics are an essential part of the development of text-generation tasks such as summarization. |
| Approach: | They propose to use top-scoring system outputs to assess the reliability of automatic evaluation metrics for text summarization. |
| Outcome: | The proposed evaluation method is based on human judgments from 25 top-scoring neural summarization systems. |
Select, Prompt, Filter: Distilling Large Language Models for Summarizing Conversations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) can be expensive to train, deploy, and use for specific natural language generation tasks. |
| Approach: | They propose a method to distill ChatGPT and fine-tune smaller LMs for summarizing forum conversations using a semantic similarity metric. |
| Outcome: | The proposed method leads to significant improvements of up to 6.6 ROUGE-2 score by leveraging sufficient in-domain pseudo-labeled data over standard KD approach given the same size of training data. |
Rehearse With User: Personalized Opinion Summarization via Role-Playing based on Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that large language models can achieve stateof-the-art performance on standard summarization benchmarks without the need for large-scale training data. |
| Approach: | They propose a personalized opinion summarization framework via LLM-based role-playing to better understand the user's personalized needs. |
| Outcome: | The proposed framework can improve the level of personalization in large model-generated summaries by taking into account user characteristics and interests while summarizing multiple product reviews. |
Automatic Text Evaluation through the Lens of Wasserstein Barycenters (2021.emnlp-main)
Copied to clipboard
| Challenge: | metric BaryScore is used to evaluate text generation based on deep contextualized embeddings. |
| Approach: | They propose to model the layer output of deep contextualized embeddings as a probability distribution rather than a vector embeddable layer. |
| Outcome: | The proposed metric outperforms other BERT based metrics and exhibits more consistent behaviour in particular for text summarization. |
Two Huge Title and Keyword Generation Corpora of Research Articles (2020.lrec-1)
Copied to clipboard
| Challenge: | Recent advances in sequence-to-sequence learning with neural networks have improved the quality of automatically generated text summaries and document keywords. |
| Approach: | They propose to use OAGSX and OAGKX datasets to analyze text summaries and document keywords. |
| Outcome: | The proposed models perform better than previous models on two large datasets . the authors hope to use the results to derive subsets of research articles from more disciplines . |
Align then Summarize: Automatic Alignment Methods for Summarization Corpus Creation (2020.lrec-1)
Copied to clipboard
| Challenge: | Summarizing text is not a straightforward task. |
| Approach: | They propose to use automated transcriptions to generate reports from automatic transcriptions as a dataset for neural summarization. |
| Outcome: | The proposed model improves on publicmeetings corpus on a dataset of aligned public meetings. |
FENICE: Factuality Evaluation of summarization based on Natural language Inference and Claim Extraction (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in text summarization have shown remarkable performance, but a significant number of summaries exhibit factual inconsistencies, such as hallucinations. |
| Approach: | They propose a factuality-oriented metric that evaluates text summarization for accuracy . they use a human annotation process to examine the accuracy of automatically generated summaries . |
| Outcome: | The proposed metric sets a new state-of-the-art on AGGREFACT, the de-facto benchmark for factuality evaluation. |
Towards Unifying Multi-Lingual and Cross-Lingual Summarization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work on multilingual summarization and cross-lingual summmarization has been limited due to their different definitions. |
| Approach: | They propose to unify MLS and CLS into a more general setting, i.e. many-to-many summarization. |
| Outcome: | The proposed model outperforms the state-of-the-art models in the zero-shot directions. |
Style-Compress: An LLM-Based Prompt Compression Framework Considering Task-Specific Styles (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prompt compression reduces inference time and costs while maintaining informativeness for different usage scenarios. |
| Approach: | They propose a framework that adapts a smaller language model to compress prompts for a larger model on a new task without additional training. |
| Outcome: | The proposed framework outperforms two baseline models in four tasks . iteratively generates and selects effective compressed prompts as task-specific demonstrations . |
Detection and Mitigation of the Negative Impact of Dataset Extractivity on Abstractive Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that extractivity can affect output extractivity and the amount of factual information (i.e. faithfulness) in abstractive summarization models. |
| Approach: | They propose to design copy labels to fix the model's copying behaviors and train the model with a copy mechanism to reduce the negative impact of high extractivity on model performance. |
| Outcome: | The proposed method outperforms several competitive baselines and shows that low extractivity can improve model performance, while higher extractivity leads to a tendency for the model to copy text continuously from the source document rather than identifying and summarizing important content. |
PAM: Paraphrase AMR-Centric Evaluation Metric (2025.findings-acl)
Copied to clipboard
| Challenge: | Current evaluation metrics for paraphrase generation are based on borrowed metrics from text-to-text tasks . this is not ideal for paraphrasing as we typically want variation in the lexicon while persisting semantics. |
| Approach: | They propose a Paraphrase AMR-Centric Evaluation Metric that uses AMR graphs extracted from the input text to evaluate paraphrases. |
| Outcome: | The proposed evaluation metric improves on different semantic textual similarity datasets on paraphrases with human semantic scores. |
FormosanBench: Benchmarking Low-Resource Austronesian Languages in the Era of Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing LLMs consistently underperform across all tasks, with 10-shot learning and fine-tuning offering only limited improvements. |
| Approach: | They introduce FormosanBench, a benchmark for evaluating LLMs on low-resource Austronesian languages. |
| Outcome: | The proposed benchmark covers three endangered Formosan languages: Atayal, Amis, and Paiwan . existing LLMs consistently underperform across all tasks, with 10-shot learning and fine-tuning offering only limited improvements. |
Can We Trust the Performance Evaluation of Uncertainty Estimation Methods in Text Summarization? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Text summarization is a key natural language generation task, but the high cost of inaccurate summaries raises concerns about the reliability of uncertainty estimation on text summarisation (UE-TS) evaluation methods. |
| Approach: | They propose a UE-TS benchmark that evaluates the uncertainty estimation capabilities of two large language models and one pre-trained language model on three datasets. |
| Outcome: | The proposed benchmark evaluates the uncertainty estimation capabilities of two large language models and one pre-trained language model on three datasets, with human-annotation analysis incorporated where applicable. |
Re-Examining Summarization Evaluation across Multiple Quality Criteria (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a number of automated evaluation metrics are evaluated by multiple quality criteria, such as relevance, consistency, fluency and coherence. |
| Approach: | They propose a method that removes the confounding variable and detects unreliable correlations. |
| Outcome: | The proposed method detects unreliable correlations between QCs and human scores . it is based on a multi-QC setup, but it fails to detect summary corruptions . |
MaCP: Minimal yet Mighty Adaptation via Hierarchical Cosine Projection (2025.acl-long)
Copied to clipboard
| Challenge: | MaCP is a new adaptation method for large foundation models that requires minimal parameters and memory for fine-tuning. |
| Approach: | They propose a method that exploits the superior energy compaction and decorrelation properties of cosine projection to improve model efficiency and accuracy. |
| Outcome: | The proposed method improves model efficiency and accuracy across a wide range of single-modality tasks including natural language understanding, natural language generation, text summarization, and multi-modalities such as image classification and video understanding. |
Curiosity-Driven Reinforcement Learning from Human Feedback (2025.acl-long)
Copied to clipboard
| Challenge: | Reinforcement learning from human feedback (RLHF) has proven effective in aligning large language models with human preferences, but often at the cost of reduced output diversity. |
| Approach: | They propose a framework that incorporates intrinsic rewards for novel states alongside traditional sparse extrinsic rewards to optimize both output diversity and alignment quality. |
| Outcome: | The proposed framework achieves significant gains in diversity on multiple diversity-oriented metrics while maintaining alignment with human preferences comparable to standard RLHF. |
DcLM: Output Length Control of Large Language Models via Dynamic Length Markers (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have limited awareness of output length, making it difficult to satisfy precise length requirements. |
| Approach: | They propose a model-agnostic approach that introduces dynamic length markers to guide length-controllable outputs. |
| Outcome: | The proposed method significantly reduces length deviation across multiple datasets. |
Behavioral Analysis of Information Salience in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models excel at text summarization, but the exact notion of salience remains unclear. |
| Approach: | They propose a framework to derive and investigate information salience in Large Language Models (LLMs) using length-controlled summarization as a behavioral probe into the content selection process. |
| Outcome: | The proposed framework derives a proxy for how models prioritize information in large language models. |
Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Survey (2025.findings-emnlp)
Copied to clipboard
| Challenge: | specialized LLMs are often limited in domain-specific applications that require specialized knowledge. |
| Approach: | They provide a comprehensive overview of four key methods to enhance large language models by integrating domain-specific knowledge. |
| Outcome: | The proposed methods are categorized into four key approaches: dynamic knowledge injection, static knowledge embedding, modular adapters, and prompt optimization. |
Summarizing Speech: A Comprehensive Survey (2025.emnlp-main)
Copied to clipboard
Fabian Retkowski, Maike Züfle, Andreas Sudmann, Dinah Pfau, Shinji Watanabe, Jan Niehues, Alexander Waibel
| Challenge: | Podcasts and other audiovisual content are becoming more and more a part of everyday communication and the digital age is changing from text to voice. |
| Approach: | They synthesize the current state of the field and highlight the need for realistic evaluation benchmarks and multilingual datasets. |
| Outcome: | The proposed frameworks are based on evaluation protocols and datasets and highlight the need for realistic benchmarks and multilingual datasets. |
Towards Visually Grounded Multimodal Summarization via Cross-Modal Transformer and Gated Attention (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for multimodal summarization often inject shallow visual features into deep models, leading to representational mismatches and weak cross-modal grounding. |
| Approach: | They propose a framework that performs text summarization and representative image selection . a deep visual processor aligns the visual encoder with the language model at corresponding depths . |
| Outcome: | The proposed framework produces more accurate, visually grounded summaries and selects more representative images. |
How Private are Language Models in Abstractive Summarization? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Effective protection of private information is essential for knowledge dissemination in sensitive domains such as medical and legal. |
| Approach: | They perform a comprehensive study of privacy risks in LM-based summarization using closed- and four-weight models of different sizes and families. |
| Outcome: | The proposed models show that they leak personally identifiable information in their summaries, compared to human-generated summary summators, which show significantly higher privacy protection levels. |
TRAC: Token-level Reward Assignment for Coherent Abstractive Summarization (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have achieved remarkable success in text summarization, but maintaining logical coherence and contextual consistency remains a pervasive challenge in long-form generation. |
| Approach: | They propose a framework that introduces a token-level reward function by integrating relative sentence gain, inter-sentence attention, and a Gaussian length penalty. |
| Outcome: | The proposed model outperforms the sequence-level baseline by 11.05% in fluency and 10.61% in Relevance. |